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HIGH-RESOLUTION  BEAMFORMING  TECHNIQUES 
PERFORMANCE  ANALYSIS 

by 

Walter  M.X.  Zimmer 


ABSTRACT 

/ 

Seven  different  beamforming  techniques  (including  both  conventional  and  high- resolution 
types)  have  been  analyzed  and  compared  in  order  to  develop  a  quantitative  relationship 
between  resolution  performance  on  the  one  hand  and  detection  and  the  accuracy  of  bearing 
estimation  on  the  other.  The  techniques  discussed  are  the  Blackman-Tukey  Conventional, 
the  Wiener  Conventional,  the  Capon  Adaptive  and  the  Maximum  Entropy  Beamformers, 
and  the  Optimal,  the  Johnson,  and  the  Schmidt  Eigenvector  Methods.  Detection  perfor¬ 
mance,  accuracy,  and  resolution  of  the  different  techniques  are  discussed  and  tabulated. 
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INTRODUCTION 


The  resolution  performance  of  conventional  beatnforming  techniques  is  limited  by  the 
length  of  the  array  (i.e.  the  array  aperture).  Therefore  high-resolution  beamforming 
techniques  may  be  achieved  only  by  extrapolating  the  measurements  (or  some  function 
of  them)  beyond  this  array  aperture.  From  mathematics  we  know  that  extrapolations 
work  reliably  only  in  the  absence  of  measurement  errors  or  noise.  In  real  life,  however, 
noise-free  measurements  are  not  possible.  The  detection  problem  is  characterized  by  a 
low  signal-to-noise  ratio,  i.e.  noise  dominates  the  measurements  and  the  signal  is  barely 
detectable.  Also,  the  accuracy  of  source  bearing  estimation  is  influenced  by  noise.  There¬ 
fore  any  extrapolation  of  the  measurements  beyond  the  array  aperture  will  likely  degrade 
the  detection  performance  of  the  beamformer  and  also  the  accuracy  of  the  source  bearing 
estimation.  The  main  purpose  of  this  report  is  to  analyze  and  compare  several  different 
beamforming  techniques  in  order  to  develop  a  more  quantitative  relation  between  reso¬ 
lution  performance  on  the  one  hand  and  detection  performance  and  accuracy  of  bearing 
estimation  on  the  other. 


1.  DESCRIPTION  OF  THE  PROBLEM 


1.1  General  problem  formulation 

Figure  1  sketches  the  following  scenario:  an  array  of  hydrophones  receives  sound  from  two 
point  sources  that  are  assumed  to  be  at  an  extreme  distance  to  avoid  complications  due  to 
changing  geometry.  This  is  equivalent  to  assuming  constant  bearing  of  the  two  sources  from 
the  array.  The  measured  data  are  processed  with  some  particular  beamforming  method 
and  the  output  is  then  displayed. 

Figure  1  shows  the  theoretical  result  of  both  a  conventional  and  a  high-resolution  method. 
The  conventional  technique  is  characterized  by  broad  main  lobes  in  source  direction  and 
more  or  less  marked  sidelobes.  The  significant  features  of  the  high-resolution  method  are 
the  sharp  peak  in  source  direction  and  the  increased  difference  between  the  peak  maximum 
and  the  mean  noise  level.  Three  questions  arise  concerning  the  performance  of  this  high- 
resolution  method: 

—  Do  narrow  peaks  indicate  better  resolution  performance? 

Is  the  bearing  estimate  more  accurate? 

Do  increased  peak-to-noise  ratios  indicate  better  detection  performance? 
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ARRAY  OF  HYDROPHONES 


CONVENTIONAL  BEAMFORMINQ 


HIOH-RESOLUTION  BEAMFORMINQ 


.  _ _  BEARING  DEVIATION 

•  PEAK  LEVEL 


PEAK  DEVIATION 


Fig.  1:  Problem  formulation;  statistical  analysis  of  passive  sonar  signal  processing. 


In  an  attempt  to  answer  these  questions,  numerous  computer  simulations  were  made.  The 
analysis  of  these  simulations  has  focused  on 

—  the  distribution  of  the  estimated  peak  level  which  yields  the  detection  performance. 

—  the  distribution  of  the  estimated  source  bearing  which  results  in  the  resolution  per¬ 
formance  and  the  accuracy  of  source  bearing  estimation. 
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In  detail,  we  will  see  how  the  peak-level  statistic  varies  as  a  function  of  input  signal-to-noise 
ratio.  Further,  we  will  see  the  degree  of  uncertainty  in  bearing  estimation,  measured  as  a 
function  of  input  signal-to-noise  ratio  and  as  a  function  of  the  separation  from  a  second, 
closely-spaced  source. 


1.2  Comments  on  the  bearnformer  output. 

When  using  a  bearnformer.  we  will  assume  that  we  wish  to  measure  the  spatial  power 
distribution  For  surh  a  case  we  apply  the  Wiener-Khinchin  theorem  and  represent  the 
power  spectrum  as  the  Fourier  transform  of  the  spatial  correlation  function,  thus: 


P(/)-  E  rke  i"*',  -!</<!.  Eq.(l) 

k=  -  oo 

If  we  know  the  entire  correlation  function,  that  is,  knowing  r*  for  all  values  of  k  ,  then 
we  can  obtain  the  power  spectrum  by  means  of  Eq.l.  However,  in  many  applications,  we 
know  (or  can  reliably  measure)  the  correlation  function  only  for  a  certain  finite  number  of 
k  values,  say  for  -p  <  k  <  p,  and  thus  we  do  not  know  the  value  of  r  for  \k\  >  p. 

To  estimate  the  power  spectrum  from  this  partial  knowledge  we  either 

—  ignore  the  long  range  correlation  which  yields  the  class  of  conventional  bearnformer, 

—  or  extrapolate  the  correlation  function  beyond  the  array  aperture  to  define  the  class 
of  high-resolution  bearnformer  [lj. 
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2.  PRESENTATION  OF  THE  BEAMFORMING  TECHNIQUES 

In  the  following  chapter  seven  different  techniques  for  beamforming  will  be  presented.  The 
techniques  can  be  conveniently  classified  into  four  different  groups: 

a.  Conventional  Beamforming  : 

-  Blackman-Tukey  method 
Wiener  method 

b.  Adaptive  Bearnforming: 

-  Capon  method 

c.  Maximum  Entropy  Bearnforming: 

-  Forward-Backward  Linear  Predictor 

d.  Orthogonal  Bearnforming 

-  Optimal  Eigenvector  method 

-  Johnson  Eigenvector  method 

-  Schmidt  Eigenvector  method 


2.1  Estimation  of  the  correlation  function. 

To  begin  we  must  first  say  some  words  about  the  spatial  correlation  function  and  its 
estimation.  According  to  the  Wiener- Khinchin  theorem  the  correlation  function  plays 
an  important  part  in  spectral  analysis.  The  same  observation  is  also  valid  in  spatial 
beamforming. 

For  a  wide  sense  stationary  process  the  (spatial)  correlation  function  is  defined  by 


Tk  M™oo  2 M  -  1*1 


Tr(n*(yy‘», 


where 


fl  is  the  next  neighbour  difference  matrix  with  dimension  (M,  M), 
=P{i  ~3  =  1), 
fT1  =  fiT, 


y 

(...) 

p{ *  -  j  =  i)  = 


is  the  measurement  vector  of  dimension  M, 
is  the  statistical  average,  and 

f  1,  if  i  -  j  =  1; 

1  0,  otherwise. 


Unfortunately  there  is  no  possibility  of  getting  the  true  correlation  function  from  real 
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measurements  as  this  would  need  infinite  spatial  and  statistical  averages.  However,  we  can 
provide  a  more  or  less  reliable  estimate  of  the  correlation  function.  Because  this  estimate 
is  always  erroneous  the  power  estimate  cannot  be  the  true  one.  It,  is  important  to  realize 
that  the  power  estimate  can  be  only  as  accurate  as  the  estimate  of  the  correlation  function. 

The  major  restriction  is  that  we  cannot  perform  an  infinite  spatial  average  (i.e.  M  — »  oo). 
This  restriction  leads  to  two  consequences: 

—  for  a  spatially  wide-sense  stationary  random  process  the  estimate  of  the  correlation 
function  will  be  noisy. 

—  because  the  maximal  correlation  lag  is  limited  by  the  array  aperture  we  cannot 
estimate  long  range  correlation. 

If  we  drop  the  notation  M  — >  oo,  the  unbiased  estimate  of  the  correlation  function  is  given 
by 

The  next  problem  is  to  compute  the  statistical  average.  In  the  case  of  beamforming  it  is 
convenient  to  replace  the  statistical  average  by  a  time  average.  This  is  justified  when  the 
sound  field  is  weakly  stationary,  not  only  in  space,  but  also  in  time.  Then  we  can  write 

1  T 

<•••>  =  ■ 

t=i 


Further  statistical  averaging  can  be  performed  by  replacing  the  data  vector  in  Eq.(2)  by  a 
forward-backward  data  matrix 


(  M\  \ 

I 

{  yi, 

Vm  ^ 

yVM  j 

vl 

y\  ) 

The  effect  of  this  is  to  replace  (yy*)  by  the  average  between  the  usual  forward  correlation 
matrix  and  the  backward  correlation  matrix  that  would  result  by  using  the  “backward” 
data  vector. 

The  reason  for  this  procedure  is  explained  by  the  following  observation: 

If  y„  represents  a  wide  sense  stationary  process,  then  its  direction-transposed  conjugated 
image  will  also  be  wide  sense  stationary  with  identical  correlation  function.  Therefore  the 
forward-backward  average  increases  the  estimation  accuracy  of  the  correlation  function  [2]. 
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2.2  Conventional  Beamforming 

Conventional  beamforming  is  characterized  by  the  fact  that  the  long  range  correlation  is 
ignored  completely  or,  equivalently,  it  is  assumed  to  be  zero,  thus 


Tk  -  0.  \k)  >  p. 

Therefore  the  power  spectrum  estimate  results  in 


P{f)  =  E  fke~2nikf. 

k^-v 

There  are  two  main  approaches  to  estimate  the  correlation  function: 

—  the  Blackman-Tukey  approach  uses  the  unbiased  finite  estimate  of  the  correlation 
function  given  in  Eq.(2). 


fk  =  rk,  |fc|  <  p. 

The  finite  aperture  of  the  array  acts  like  a  rectangular  window  on  the  correlation 
function.  Therefore  the  true  power  spectrum  can  be  achieved  for  short  range  cor¬ 
related  processes  (i.e.  wide  angle  noise),  but  the  spectrum  is  heavily  oscillating  in 
situations  where  the  long  range  tail  of  the  correlation  function  is  of  importance  (point 
sources). 

—  The  Wiener  approach  reduces  these  oscillations  and  always  ensures  positive  power 
estimates  by  using  a  biased  estimate  of  the  correlation  function.  This  estimate  is 
given  by  multiplying  the  unbiased  estimate  with  a  triangular  (Bartlett)  window: 


fk  =  {  1  - 


1*1  <  P- 


Besides  these  power  estimators  that  are  based  on  the  correlation  function,  we  have  the 
Periodogram  Method  of  Schuster.  This  method  was  the  first  important  application  of  the 
Fourier  Spectral  Theorem.  In  this  method  the  power  spectrum  is  defined  as  the  squared 
value  of  the  Fourier  Transform  of  all  the  data: 

M 

Hi)  lE^  2*'"/|2- 

n  ! 


SACI.ANTCKN  SK-KM 


1 


ll  is  easy  to  sec  that  tins  method  transfoirns  to  the  Wiener  approach  by  taking  the  expec¬ 
tation  value  on  both  sides 

It  should  be  noted  that  t  h*  ■  «  \  pres*,  ion .  <  nm  ent  ion  a  I  beamforming,  is  used  in  a  wider  sense 
than  usual  The  normal  usage  regards  <>nlv  the  periodogram  and  the  Wiener  approach 
as  conventional  beamformers  For  systematu  reasons,  however,  the  Blackman-Tukey  ap¬ 
proach  also  will  be  called  conventional  ber  ause  it  is  the  unbiased  implementation  of  the 
Wiener  -  Kit  inch  in  t  heorem. 


2.3  Adaptive  Beamforming 

One  of  the  early  high- resolution  methods  is  the  adaptive  beamforming  technique  of  Capon 
{ 3 1 .  This  approach,  sometimes  called  maximum  likelihood  method,  extends  the  correlation 
function  in  such  a  way  that  interferences  from  all  other  than  the  steering  directions  become 
minimum.  This  is  equivalent  to  minimizing  the  beainformer  output  under  the  constraint 
that  the  response  in  a  given  direction  is  kept  constant. 

Using  the  Lagrange  multiplier  method  we  can  write  the  unconstrained  optimization  pro¬ 
cedure  as  follows: 


where 


d'Sd  •  n(d'  tr  c) 


.S'  yy'  is  the  cross  correlation  matrix, 
d  is  the  optima!  steering  vector, 

p  is  some  Lagrange  parameter, 

w  -  w(f)  is  the  look  direction, 

c  is  some  constant. 


The  resulting  power  spectrum  estimate  is  given  by 


ir'  .S'  1  tr  ' 


where  N  p  is  the  desired  aperture  of  the  beainformer. 

There  is  no  known  closed-form  expression  to  show  how  the  correlation  function  r*  is  esti¬ 
mated  within  and  outside  the  physical  array  aperture.  However,  some  indication  for  this 
behaviour  may  be  found  in  [1  .  In  this  report  it  is  only  of  importance  that  the  Capon 
method  modifies  the  measured  part  of  the  correlation  function  and  tries  to  produce  a 
smooth  extrapolation. 
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2.4  Maximum  Entropy  Beamforming 

In  this  paragraph  the  maximum  entropy  principle  will  be  discussed  as  a  method  yielding 
high  resolution.  This  principle  is  well  known  in  physics;  for  example  it  is  the  basis  of 
statistical  mechanics. 

Before  we  formulate  the  maximum  entropy  principle  we  have  to  define  entropy: 

Entropy  is  a  measure  of  the  number  of  distinct  possible  ways  of  achieving  a 
probability  distribution. 

A  high  value  of  entropy  therefore  means  that  the  probability  distribution  has  an  extensive 
statistical  support.  The  maximum  entropy  principle,  a  plausible  consequence,  says: 

Inferences  based  on  incomplete  information  should  be  drawn  from  whichever 
probability  distribution  has  the  maximum  entropy  permitted  by  the  available 
information. 

As  an  example,  if  we  only  have  knowledge  about  second-order  statistics  the  maximum 
entropy  principle  results  in  the  gaussian  probability  distribution  as  a  basis  for  inferences. 

The  maximum  entropy  principle  produces  the  following  consequence: 

Unless  there  are  further  constraints  which  are  not  revealed  in  the  statement 
of  the  problem,  the  great  majority  of  the  true  power  spectra  will  be  close  to 
the  maximum  entropy  estimate,  because  the  great  majority  of  all  possible 
spectra  have  that  property.  Conversely  if  the  maximum  entropy  estimate 
turns  out  to  be  significantly  in  error,  then  we  have  statistically  significant 
evidence  for  the  existence  of  a  systematic  effect  separating  us  from  the  desired 
solution. 

The  maximum  entropy  method  is  reliable  in  the  sense  that  it  cannot  show  any  details  for 
which  there  is  no  evidence  in  the  data  [4]. 

Assuming  that  we  know  the  expectation  value  of  the  correlation  function  r  (our  variable 
of  interest),  then  the  maximum  entropy  is  given  by  |4,  5] 


X  !<>gZ  *  XTr, 


where 

Z  is  the  partition  function, 

r  is  the  expectation  value  of  the  correlation  function, 
X  is  a  vector  of  Lagrange  parameters. 
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The  Lagrange  parameter  in  this  equation  can  be  interpreted  simply  as  the  potential  of 
the  corresponding  data.  Redundant  data,  which  by  definition  do  not  contribute  to  the 
entropy,  are  at  zero  potential.  On  the  other  hand,  highly  relevant  data  are  those  without 
which  our  estimate  would  be  very  different;  they  have  a  large  potential  and  their  absence 
would  greatly  lower  the  entropy. 

This  rather  abstract  treatment  of  the  maximum  entropy  principle  was  introduced  to  show 
that,  theoretically,  this  principle  is  very  appealing.  However  we  will  see  later  on  that  the 
optimal  behaviour  cannot  be  achieved  in  reality. 

The  maximum  entropy  spectral  estimate  is  given  by 


Hi)  YL  he'2wtkf  =  ~  , 

a— ^  n  *  in 


where  N  »  p  is  the  desired  aperture  of  the  beamformer  and  the  vector,  a,  may  be  estimated 
via  the  relation 


a‘  Ra  n{a' c  -  1)  — *  min 


R  V  rk n\ 

*  r 

c  (1,0 . 0)r, 

p  is  some  new  Lagrange  parameter. 


The  power  spectrum  estimate  is  then  given  by 


pm  I  I 

i  r 1  R  1  w  i 

The  maximum  entropy  spectrum  is  consistent  with  the  Wiener-Khinchin  theorem  because 
it  is  only  the  closed  form  solution  in  which  the  unknown  part  of  the  correlation  function 
is  replaced  by  the  estimate  based  on  the  maximum  entropy  signal  distribution.  The  ex¬ 
trapolation  of  the  correlation  function  is  consistent  with  the  data  and  introduces  no  new 
information  into  the  spectrum. 

However,  the  power  spectrum  estimate  is  optimal  in  the  maximum  entropy  sense  only 
if  the  correlation  matrix  is  of  Toeplitz  form.  In  reality  this  may  only  be  achieved  with 
infinite  temporal  or  statistical  average  Therefore,  any  implementation  of  a  maximum 
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entropy  spectrum  estimator  will  be  sub-optimal  and  sometimes  have  unpredictable  results. 
To  achieve  acceptable  results  Nuttall  l‘2l  proposed  to  use  a  forward- backward  averaged 
estimate  of  the  correlation  matrix.  A  further  ad  hoc  assumption  is  given  by  the  limitation 
of  the  maximum  measured  correlation  length  to  a  value  less  than  the  array  aperture.  This 
results  in  additional  statistical  stability.  This  implementation  of  the  maximum  entropy 
principle  is  also  called  the  forward-backward  linear  predictor  and  will  be  used  in  this 
report. 


2.5  Orthogonal  Beamforming 

The  orthogonal  beamforming  principle  is  based  on  the  following  observations: 

—  'he  correlation  matrix  is  Hermitian  and  therefore  all  its  eigenvectors  are  mutually 
orthogonal; 

—  if  the  correlation  matrix  is  of  rank  A/  and  the  signal  alone  correlation  matrix  is  of 
rank  K  <  M ,  then  A /  -  K  noise  (signal  free)  eigenvalues  exist; 

—  if  the  noise  is  white  and  the  correlation  matrix  exactly  known,  then  the  noise  eigen¬ 
vectors  are  orthogonal  to  the  signal  bearing  vectors. 

if  the  correlation  matrix  is  known  but  the  noise  is  not  white,  then  there  is  orthogo¬ 
nality  between  the  noise  eigenvectors  and  the  signal  bearing  vectors  obtained  after 
the  appropriate  ti  ausformation  (given  by  the  spatial  pre-whitening  of  the  noise)  is 
applied. 

The  fundamental  eigenvector  method  is  due  to  Pisarenko  (e.g.(6()  and  may  be  described 
as  a  minimum  interference  method,  where  the  solution  vector  d  is  found  via  the  constant 
norm  optimization  procedure, 

d '  Hd  t  nd' d  min, 

with  the  solution 

P  Amin 

as  the  minimum  eigenvalue  of  R,  and 

d 

as  the  eigenvector  associated  with  A„,,„  (also  called  minimum  eigenvector). 

To  have  a  beamformer  output  a  pseudo-power  estimate  may  be  defined  using  a  formula 
similar  to  the  maximum  entropy  spectrum  estimate: 
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P(f) 


12  ' 


where  xr  «’(/)  is  the  steering  vector. 

From  this  formula  we  see  that  any  steering  vector  parallel  to  a  source  eigenvector  will 
result  in  a  sharp  peak  of  P(f). 

However,  the  minimum  eigenvector  beamforiner  is  not  recommended  for  the  following 
reasons: 


—  In  real  applications  the  cross-correlation  matrix  will  be  estimated  via  a  finite  av¬ 
erage.  Consequently  all  eigenvalues  and  in  particular  the  minimum  eigenvalue  are 
estimated  only  within  finite  error  bounds.  This  uncertainty  may  be  negligible  for 
large  eigenvalues  but  plays  an  important  role  for  the  minimum  eigenvalue.  As  the 
minimum  eigenvalue  is  relatively  uncertain,  the  corresponding  eigenvector  is  nearly 
unpredictable. 

—  In  all  cases  where  the  number  of  sources  K  is  significantly  less  than  the  number 
of  hydrophones  M  ,  or  more  precisely,  if  K  <  M  -  1,  then  not  only  the  signal 
eigenvectors  but  also  M-l-K  noise  eigenvectors  will  be  orthogonal  to  the  minimum 
eigenvector.  This  means  that  not  only  possible  source  directions  may  peak  but  also 
the  residual  “noise  directions”. 


To  overcome  these  problems  one  must  first  estimate  the  number  of  noise  eigenvalues; 
secondly,  one  may  try  to  combine  the  different  noise  eigenvalues  and  eigenvectors  in  some 
way  to  produce  satisfactory  results.  The  first  part,  namely  the  estimation  of  the  number 
of  noise  eigenvalues,  is  a  difficult  decision  problem.  Here,  for  this  report,  we  evade  the 
problem  by  assuming  that  the  number  of  noise  eigenvectors  is  known  a  priori.  Concerning 
the  combination  of  the  different  noise  eigenvalues  and  eigenvectors,  there  are  two  major 
approaches  which  are  outlined  next. 


Recently  Kumaresan  and  Tufts  [7]  proposed  a  simple  concept  for  an  optimal  orthogonal 
beamformer.  They  replaced  the  minimum  eigenvector  by  an  optimal  linear  combination 
of  all  noise  eigenvectors  according  to  the  following  optimization: 


Model 


M-K 

d  =  Yi  b,v' 

i=l 


v,  is  the  t-the  noise  eigenvector 
6,  is  a  constant  to  be  estimated 
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Optimality  criterion 


\d\2  — ►  min 


Constraint 


d' c  —  1,  c  is  a  constant 


Solution 


bt 


v,  c 


M-K  |  .  12 

E,=i  kcl2 


A  second  approach  combines  the  different  power  estimates  of  all  possible  noise  eigenvector 
solutions 


1 

W) 


M-K 

- 


t=i 


where  6,  is  a  constant  to  be  estimated,  and 


Pi 


where 

A,  is  the  t  -  th  noise  eigenvalue, 
vt  is  the  corresponding  eigenvector,  and 
w  —  w(f)  is  the  steering  vector. 

Parameter  6,  has  two  known  implementations: 

,  _  j  1,  Johnson  and  DeGraaf  [8]; 

6‘  ~  \  A„  MUSIC  of  Schmidt  [9). 
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3  THEORETICAL  RESULTS 

3.1  Summary  of  the  beamforming  techniques 

Before  the  analysis  of  the  different  beamformers  is  presented  a  summary  of  the  “pseudo” 
power  estimates  is  given.  The  expressions  correspond  also  to  the  formulas  programmed 
into  the  computer. 

a)  Notation 

Correlation  Matrix 


s  =  ~=  ^2(yty't  +  mi ),  y  =  (yM,...,y  i)‘ 

t= i 

S  is  the  spatial  averaged  correlation  matrix. 


Correlation  Function 

h  =  (M  -  \k\)~lTr{Sflk) 


Eigenvector  Equation 


Sv,  ~  Atv, 


where 

A,  is  the  t-th  eigenvalue  of  5 
v,  is  the  eigenvector  to  A, 


Steering  vector 

MI)  =  e~2r,kf,  f=  ^  cos(w) 

where 

d  is  the  array  spacing, 

A  is  the  signal  wave  length, 
w  is  the  steering  angle. 


b)  Conventional  Beamforming 

P(f)  2Re(rw{f))/M,  f„  -  r0/2 
Blackman-Tukey 


h  -  rkp(\k\  <  M) 
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Wiener 


fk  =  rk( 1 


£!)p(I*I  <  M ) 


c)  Adaptive  Beamforming 


Hf) 


i 


xv'S~lxv 


d)  Maximum  Entropy  Beamforming 


Hf)  = 


S  1  c 
cTS~l  w 


2 


e)  Orthogonal  Beamforming 

Optimal  Eigenvector  Method 


P{f)  = 


cT(Hl iiK  Vivr)w 


2 


Harmonic  Averaging  Technique 


P(f)  = 


1 


E 


M-K 
»  =  1 


XV 


Johnson 

MUSIC  of  Schmidt 


3.2  Assumptions 

The  results  presented  in  this  chapter  are  based  on  the  exact  knowledge  of  the  cross  corre¬ 
lation  function: 

K 

rk  =  o2np(k  --  0)  +  Y^°*S' 

i  l 


where 

is  the  noise  spectral  power; 
<7,2  is  the  signal  spectral  power; 
S,  is  the  signal  bearing  vector; 
K  is  the  number  of  signals. 
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The  beamformer  outputs,  derived  by  using  the  different  techniques,  are  presented  in  Fig¬ 
ures  2  to  8.  The  results  are  based  on  the  following  parameter  values: 


Number  of  hydrophones:  A/ 

-  32 

Order  of  linear  predictor:  p 

^  24 

Number  of  signals:  K 

=  2 

Noise  power:  a ^ 

-  0  dB 

Signal  to  noise  ratios:  of  jo\ 

-  0  dB 

=  -10  dB 

Signal  bearings:  Si 

-  75/128  -  65.56° 

s2 

-  80/128  =  77.36° 

3.3  Conventional  Beamforming 

Outputs  for  the  conventional  beamformer  are  presented  in  Figures  2  and  3. 

The  Blackman- Tukey  method  as  shown  in  Fig.  2  is  characterized  by  an  oscillating  power 
spectrum  and  relatively  high  sidelobe  levels  that  tend  to  disguise  the  second  source  (which 
has  a  signal-to-noise  ratio  of  - 10  dB).  The  Wiener  approach  shows  positive  power  estimates 
as  well  as  lower  sidelobe  levels.  Therefore  the  second,  weaker  source  is  clearly  visible. 
However,  the  mainlobes  are  enlarged  compared  to  those  of  the  Blackman-Tukey  method. 


3.4  Adaptive  Beamforming 

Figure  4  presents  a  typical  output  of  the  Capon  beamformer.  The  two  sources  can  be 
seen  very  clearly.  No  significant  sidelobes  are  present  in  the  figure.  The  signal-to-noise 
ratio  in  source  direction  corresponds  to  the  value  measured  with  the  conventional  Wiener 
beamformer.  As  a  consequence  the  Capon  beamformer  may  be  used  as  a  power  estimator. 


3.5  Maximum  Entropy  Beamforming 

Figure  5  gives  the  output  of  the  Maximum  Entropy  Method.  Two  interesting  features 
may  be  observed;  first,  the  output  signal-to-noise  ratio  is  apparently  twice  those  of  the 
previously  presented  methods.  Second,  there  are  sidelobes  in  the  form  of  oscillations.  This 
is  a  strong  indication  that  the  extrapolation  of  the  correlation  function  has  not  been  made 
in  an  analytic  way,  thus  verifying  Lunde  and  Zimmer  [l]. 


3.6  Orthogonal  Beamforming 

The  beamformer  outputs  of  the  eigenvector  techniques  are  given  in  Figures  6,  7  and  8.  In 
these  figures  we  see  two  peaks  of  equal  level  independent  of  the  input  signal-to-noise  ratio. 
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STEERING  ANGLE  (DEG  ) 


Fig.  5:  Maximum  Entropy  Beamforming;known  correlation  function 


STEERING  ANGLE  ( DEG  ) 


STEERING  ANGLE  ( DEG  ) 


Fig.  8:  Schmidt  Eigenvector  Method;  known  correlation  function. 

These  features  indicate  clearly  that  orthogonal  beamformers  are  direction  finders  and  not 
power  estimators.  Further,  the  two  techniques,  which  are  based  on  harmonic  averaging, 
show  a  dynamic  range  of  about  37  dB;  this  range  is  probably  due  to  computer  limitations. 
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4  DETECTION  PERFORMANCE 

This  section  presents  the  analysis  of  the  detect  ion  performance  of  the  different  beamforming 
methods  described  previously. 

1.1  Preliminaries 

A  clear  indication  of  detection  performance  was  derived  from  a  series  of  simulations.  The 
scenario  consisted  of  a  single  source  of  while  noise.  A  hydrophone  cross-correlation  matrix 
was  estimated  by  means  of  .V.tl  10  averages.  After  beamforming,  the  peak  level  of 
the  local  maximum  nearest  to  the  direction  of  interest  was  measured  (Figure  9).  For  each 
given  signal-to-noise  ratio  this  experiment  was  repeated  1000  times  to  provide  sufficient 
data  for  statistical  analysis.  To  reduce  the  time  requirements  of  the  simulations  all  seven 
methods  were  applied  to  the  same  correlation  matrix. 


PEAK  LEVEL 

RELEVANT 
PEAK  LEVEL 


Fig. 9:  Definition  of  a  local  maximum. 

'The  peak  level  is  easily  found  when  the  signal  is  present.  There  are  three  possibilities  for 
defining  the  distribution  of  noise-alone  level: 

f  ixed  beam  noise  distribution:  distribution  of  the  noise  level  in  the  direction  of 
interest . 

Local  noise  distribution:  distribution  of  the  peak  level  of  the  local  maximum  nearest 
to  the  direction  of  interest. 
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Globa!  noise  distribution:  distribution  of  the  peak  level  of  the  global  maximum. 

Of  these  possibilities  only  the  local  noise  distribution  is  consistent  with  the  assumption 
that  the  source  direction  is  dete  rmined  In  estimating  the  local  maximum  nearest  to  the 
direction  of  interest.  Therefore  this  definition  has  been  used  throughout  this  report. 

1.2  Detection  probability 

Appendix  B  provides  a  summary  of  the  simulations  that  gave-  the  detection  performance. 
The  cumulative  probability  of  the  peak-level  estimation  is  plotted  against  its  estimated 
level.  With  gaussian  scaling  of  the  probability  axis  (ordinate)  and  the  peak  level  (abscissa) 
in  dB.  a  straight  line  represents  a  log-normal  distribution  of  the  peak-level  estimates. 

From  the  cumulative  distribution  of  the  peak-level  estimation  we  can  easily  deduce  the 
detection  probability  (Figure  10).  First  we  plot  the  noise  distribution  (no  signal)  so  that 
the  false  alarm  probability  can  be  found  directly  from  the  ordinate.  For  every  false  alarm 
probability  the  abscissa  gives  the  threshold  to  use  to  get  the  detection  probability  from 
the  signal  curves.  No  assumptions  are  made  about  the  statistics,  e.g.  the  peak  levels  of 
the  local  maxima  may  or  may  not  have  a  normal  or  log-normal  distribution.  The  selection 
of  log-normal  scaling  was  only  made  for  plotting  convenience. 

Figure  10  provides  detection  probability  curves  for  each  of  the  seven  beamforming  tech¬ 
niques  discussed  in  this  report.  The  assumed  false  alarm  probability  P/a  was  1.35  x  10“3. 
The  Wiener  Conventional  Beamformer  and  Johnson  Eigenvector  methods  provided  the 
highest  detection  probabilities;  the  Maximum  Entropy  and  Optimal  Eigenvector  methods 
provided  the  lowest. 


From  theoretical  analysis  one  could  assume  that  all  eigenvector  techniques  are  poor  as 
detectors  because  they  are  direction  finders,  not  power  estimators.  Therefore  the  meth¬ 
ods  of  Johnson  and  Schmidt  performed  surprisingly  well,  probably  because  the  theory 
of  orthogonal  beamformers  assumes  that  the  signal-bearing  vectors  are  parallel  to  the 
signal  eigenvectors  and  orthogonal  to  the  noise  eigenvectors.  This  theory  requires  the 
cross-correlation  matrix  to  be  estimated  by  an  infinite  average;  this  is  not  done  in  reality. 
As  a  consequence  the  orthogonality  between  signal-bearing  vectors  and  noise-eigenvectors 
breaks  down.  Apparently  the  harmonic  averaging  of  the  Johnson  and  Schmidt  techniques 
uses  this  effect  to  generate  and  stabilize  an  angle  between  the  signal-bearing  vector  and  the 
noise  subspace  that  depends  on  the  input  signal-to-noise  ratio  of  the  sound  source.  The 
Optimal  Eigenvector  technique  cannot  show  such  a  feature  because  of  its  linear  averaging. 

4.3  Accuracy  of  bearing  estimation 

Figure  1  1  is  a  plot  of  the  Standard  Deviation  of  the  bearing  estimation  as  a  function 
of  input  signal-to-noise  ratio  for  all  seven  techniques.  The  Standard  Deviation  has  been 
estimated  by  the  relation: 
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SIGNAL-TO-NOISE  RATIO  (dB) 


Fig.  10:  Probability  of  detection  as  a  function  of  input  signal-to-noise  ratio. 

W:  Wiener  Conventional  Beamfornier;  B:  Bla<  kman-Tukey  Conventional  Beamformer;  C: 
Capon  Adaptive  Beamformer;  M:  Maximum  Entropy  Method(Forward-  Backward  Linear 
Predictor);  O:  Optimal  Eigenvector  Method;  J:  Johnson  Eigenvector  Method;  S:  Schmidt 
Eigenvector  Method 


STANDARD  DEVIATION 
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Fig  11:  Standard  deviation  of  bearing  estimation  as  a  function  of  input  signal-to-noise 
ratio. 

VV:  Wiener  Conventional  Bearnformer;  B  Blackman  l  ukey  ( lonveni ional  Beamformer;  C: 
Capon  Adaptive  Beamformer:  M  Maximum  Entropy  Method  (Forward-  Backward  Linear 
Predictor):  0:  Optimal  Eigenvector  Method,  J  Johnson  Eigenvector  Method;  S:  Schmidt 
Eigenvector  Method 
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♦  m'(I)  ♦„'(  ■) 

(1 

•> 

where  <t>  ( j  )  is  the  cumulative  (list  nl>ut  ion  ol  the  hearing  estimation. 

This  formula  assumes  a  gaussian  (list rihut ion  for  the  hearing  estimates.  This  is  a  good 
approximation  tor  signal-to-noise  ratios  that  are  not  too  low.  1  he  actual  values  of  the 
standard  deviation  have  been  normalized  with  respect  to  the  reciprocal  aperture  and  are 
plotted  in  log  scale 

The  results  indicate  that  the  Wiener  technique  with  the  lowest  standard  deviation  of  all 
seven  techniques,  performs  l.est  The  Optimal  Eigenvector  and  the  Maximum  Entropy 
techniques  have  the  worst  performance  (as  they  did  for  the  detection  probability). 

At  first  glance  it  seems  surprising  that  the  high-resolution  methods  perform  so  badly.  But  a 
closer  look  at  these  methods  provides  a  simple  explanation  related  to  the  fact  that  the  high- 
resolution  methods  extrapolate  the  correlation  function  beyond  the  array  aperture.  The 
sharper  the  peak  of  a  power  spectrum  the  longer  the  correlation  function  must  be.  If  the 
correlation  function  is  not  known  exactly  within  the  array  aperture,  then  the  extrapolation 
may  be  unstable  and  in  most  cases  will  amplify  the  influence  of  the  noise  on  the  estimation 
performance. 
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5  RESOLUTION  PERFORMANCE 

5.1  Scenario 

To  study  the  resolution  capabilities  of  the  different  beamfonners  the  following  simulations 
were  repeated  1000  times:  two  closely  spaced  point  sources  in  white  noise  are  assumed  to 
have  equal  signal-to-noise  ratio  of  0  dB  and  given  varying  separations.  The  directions  of 
interest  are  given  by  the  simulated  source  directions.  The  datum  is  ignored  if  a  single  local 
maxi-  mum  is  nearest  to  both  directions  of  interest  and  therefore  no  resolution  is  observed. 
The  sources  are  considered  as  being  resolved  when  there  are  two  local  maxima. 

Three  quantities  are  of  interest: 

—  the  bias  of  the  bearing  estimation, 

—  th  deviation  of  the  bearing  estimation, 

—  the  probability  of  resolution. 


5.2  Resolution  probability 

Appendix  B  presents  the  statistics  of  the  two  source-bearing  estimation  experiments.  The 
probability  axis  (ordinate)  is  again  scaled  such  that  a  normal  distribution  results  in  a 
straight  line.  The  abscissa  measures  the  separation  in  units  of  1/aperture.  The  nominal 
source  separation  at  which  the  simulation  has  been  carried  out  is  the  curve  parameter. 

Figure  12  presents  the  main  results  of  the  resolution  experiments.  Here  the  probability 
that  two  sources  are  resolved  is  plotted  against  the  separation  of  these  two  sources.  The 
probability  is  given  in  gaussian  scaling  and  the  source  separation  is  measured  in  beam 
number  units  (of  1/aperture). 

As  expected,  the  Wiener  conventional  beamformer  performed  poorest.  On  the  other  hand 
the  high-resolution  techniques  were  able  to  resolve  the  two  sources  even  when  they  had  a 
small  separation. 

Except  for  the  Optimal  Eigenvector  technique,  all  the  high-resolution  methods  have  a 
rather  similar  performance.  We  can  therefore  deduce  that  at  least  for  simulated  data  the 
effective  beamformer  aperture  of  these  high-resolution  techniques  is  of  the  same  order  of 
magnitude  for  all  of  them. 


t 
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Fig.  12:  Probability  of  resolution  as  a  function  of  two  source  separation  (measured  in  units 
of  1 /aperture) 

W:  Wiener  Conventional  Beamformer;  B:  Blackman-Tukey  Conventional  Beamformer;  C: 
Capon  Adaptive  Beamformer;  M:  Maximum  Entropy  Method  (Forward-  Backward  Linear 
Predictor);  O:  Optimal  Eigenvector  Method;  J:  Johnson  Eigenvector  Method;  S:  Schmidt 
Eigenvector  Method 
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6  SUMMARY 


This  report  presents  a  statistical  analysis  of  seven  high-resolution  beamforming  tech¬ 
niques.  A  comparison  has  been  made  between  the  conventional  techniques  on  the  one 
hand  ( Wiener, D-T)  and  conceptionally  different  high- resolution  methods  on  the  other 
hand.  These  high-resolution  techniques  have  been  selected  from  three  groups:  Adap¬ 
tive  (Capon).  Maximum  Entropy  (Max-Ent)  and  Orthogonal  (Opt-Eig., Johnson, Schmidt) 
Beamforrners. 

Table  1  compares  the  statistical  performance  of  the  seven  techniques  with  respect  to  de¬ 
tection,  accuracy  and  resolution. 


TABLE  1 

Performance  Statistics 


Method 

Detection 

Wiener 

1.0 

B-T 

0.9 

Capon 

0.9 

Max*-Ent 

0.7 

Opt-Eig 

0.4 

Johnson 

1.0 

Schmidt 

0.9 

Accuracy  Resolution 


1.0 

1.0 

0.8 

1.3 

0.7 

1.8 

0.6 

2.0 

0.4 

3.0 

0.8 

2.1 

1.0 

2.1 

This  table  is  based  on  the  following  assumptions: 

—  the  detection  and  resolution  performance  measures  can  be  taken  as  corresponding  to 
the  50%  point  in  the  probability  curves.  The  detection  performance  is  measured  in 
dB  and  the  resolution  performance  is  measured  in  multiple  apertures. 

—  the  accuracy  measure  can  be  represented  by  the  standard  deviation  at  an  input  signal 
-to-noise  ratio  of  -10  dB  (equivalent  to  a  conventional  output  signal-to-noise  ratio 
of  5  dB). 

—  the  actual  values  can  be  related  to  the  conventional  Wiener  beamformer  to  yield  a 
quantitative  comparison  of  the  different  techniques. 

From  Table  1  we  can  conclude  that  improved  resolution  performance  of  the  high-resolution 
methods  is  gained  at  the  the  cost  of  a  more  or  less  significant  decrease  in  accuracy  and 
detection  performance.  In  particular  the  high-resolution  Maximum  Entropy  and  Optimal 
Eigenvector  techniques  show  this  behaviour  clearly.  These  techniques  may  be  understood 
as  vector  techniques  in  which  the  power  estimate  is  based  on  the  computation  of  a  single 
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vector.  The  other  high-resolution  techniques  u'apon.  Johnson,  Schmidt)  may  he  called 
matrix  techniques  because  they  use  the  hydrophone  cross-correlation  matrix  (or  a  part  of 
it)  to  get  the  power  estimate  These ’hree  matrix  techniques  show  a  detection  performance 
only  slightly  decreased  wit  h  respe<  t  to  t  he  reference  tec  hnique  (  \\  inner).  This  is  surprising, 
at  least  for  the  orthogonal  beamlormers  ot  Johnson  and  Schmidt;  these  two  methods 
are  designed  to  be  direction  finders,  not  power  estimators.  For  perfectly  known  cross¬ 
correlation  matrices  they  would  indicate  the  source  direction  with  infinitely  large  spikes. 
However,  we  have  only  a  non-perfec  t  estimate  o|  the1  correlation  matrix,  yielding  statistical 
fluctuations  of  the  peaks  of  the  estimator  Therefore  a  paradox  arises,  namely  that  the 
poor  statistics  based  on  finite  averaging  of  t he-  c  ross-correlation  matrix  enable  us  to  use 
the  orthogonal  beamlormers  as  detectors. 

Another  important  conclusion  from  Table  1  relates  to  the  accuiacy  values.  None  of  the 
techniques  is  more  accurate  than  the  conventional  Wiener  technique.  The  consequence 
is  clear:  sharp  peaks  in  the  beamformer  output  do  not  indicate  more  accurate  bearing 
estimations;  they  are  only  necessary  to  achieve  higher  resolutions.  Again  the  two  vector 
methods  (Max-Ent;  Opt-Eig)  show  the  worst  performance  while  the  difference  between  the 
matrix-based  orthogonal  beamformers  (Johnson,  Schmidt)  is  less  significant. 

Concerning  the  resolution  performance  the  three  matrix  techniques  (Capon,  Johnson, 
Schmidt),  these  do  not  perform  as  well  as  the  vector  techniques.  However,  all  high- 
resolution  techniques  show  an  increased  effective  aperture  that  is  at  least  1.8  times  greater 
than  the  conventional  (real)  aperture. 

On  this  basis  one  can  say  that  the  high-resolution  methods  showed  increased  resolution 
performance;  however,  in  cases  where  there  is  no  resolution  problem  they  did  not  provide 
more  accurate  bearing  estimates  than  tin  conventional  methods.  Because  any  increased 
resolution  is  based  on  the  extrapolation  of  the  hydrophone  cross-correlation  function,  the 
influence  of  the  noise  on  such  an  extrapolation  must  be  controlled  very  carefully. 
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APPENDIX  A 


Further  comments  on  the  beamformer  output 


The  definition  of  beamformer  output  as  used  throughout  this  report  is  not  free  of  problems. 
In  particular  the  construction  of  high-resolution  beamformers  would  appear  to  emphasize 
the  direction-finding  behaviour  rather  than  the  power  estimation  performance.  Suppose  we 
drop  the  requirement  that  the  high-resolution  beamformer  should  be  a  power  estimator. 
What  can  we  do  to  get  not  only  the  source  bearing  but  also  the  power  estimates?  We  could 
take  the  estimated  source  directions  and  insert  them  into  an  optimal  power  estimation 
scheme  such  as  the  maximum  likelihood  parameter  estimation  technique.  However,  this 
approach  only  solves  the  problem  of  the  decreased  detection  performance  of  the  high- 
resolution  methods.  To  improve  the  accuracy  of  the  bearing  estimation  we  have  to  select 
a  nonlinear  maximum  likelihood  approach  which  now  has  the  bearing  of  the  different 
sources  as  additional  unknown  parameters.  As  a  consequence  the  computational  workload 
will  increase  enormously.  In  this  case  the  high-resolution  direction  finder  would  act  as  a 
preprocessor  for  a  multivariant  parameter  estimation  technique  [10].  It  is  not  yet  clear 
under  which  conditions  the  trade-off  between  optimal  performance  and  computational 
burden  may  favour  this  parameter  estimation  approach  to  high-resolution  beamforming. 
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APPENDIX  B 


Summary  of  the  Single-Source  simulations 

In  this  appendix  the  results  of  the  single-source  simulations  for  the  seven  different  beam- 
forming  techniques  are  summarized  by  plots  of  cumulative  distributions.  In  ail  of  the  plots 
the  measured  peak  level  is  plotted  along  the  abscissa  in  dB  and  the  cumulative  probability 
is  plotted  along  the  ordinate  according  to  a  gaussian  scaling.  The  gaussian  error  function 
has  been  approximated  by 


<f>(x)  ---  (l  -f  y  l  -■  exp(-2x2/7r)/2 

where  the  plus  sign  is  selected  for  x  >  0  and  the  minus  sign  for  x  <  0.  The  error  made  by 
this  approximation  is  given  in  the  following  figure. 


For  each  plot  (Figs.  Bl  to  B7)  the  signal  curves  rise  with  increasing  peak-level  and  give 
the  probability  of  the  peak  level  being  less  than  the  value  at  the  abscissa.  The  noise  curve 
on  the  other  hand  is  decreasing  with  increasing  abscissa  and  indicates  the  probability  of 
the  peak  level  being  greater  than  the  value  of  the  abscissa.  This  value  also  corresponds  to 
the  false  alarm  probability. 

Figures  B4  (Maximum  Entropy)  and  B5  (Optimum  Eigenvector)  are  incomplete  because 
there  was  insufficient  data.  These  techniques,  even  on  a  logarithmic  scale,  have  such  a  wide 
distribution  that  1000  repetitions  are  not  sufficient  to  produce  adequate  statistical  data. 
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The  Johnson  Eigenvector  method  (Fig.  B6)  yet  again  shows  well-behaved,  nearly  log¬ 
normal  distributions. 

The  Schmidt  Eigenvector  method  (Fig.  B7)  provides  an  unexpected  feature.  The  statis¬ 
tics  strongly  indicate  that  the  distributions  have  a  lower  cut-off  at  0  dB.  Of  1000  peaks, 
none  was  measured  below  this  value.  The  construction  of  this  algorithm  may  explain  this 
phenomenon;  the  true  noise  eigenvalues  are  replaced  by  unity  (i.e.,  0  dB). 

The  Schmidt  Eigenvector,  the  Maximum  Entropy  and  the  Optimal  Eigenvector  methods 
show  that  basing  the  performance  analysis  only  on  first  and  second  order  statistics  rather 
than  on  the  complete  probability  distribution  can  be  misleading. 
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SIGNAL  TO  NOISE  RATIO  (dB) 

Fig.  BS:  Capon  Adaptive  Beamformer 


Probability  of  the  peak-level  of  a  single  source.  Parameter  is  input  signal-to-noise  ratio. 
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NOISE 

SIGNAL 

SIGNAL  TO  NOISE  RATIO  (dB) 

Fig.  Bj:  Maximum  Entropy  Method 

Probability  of  the  peak-level  of  a  single  source.  Parameter  is  input  signal- to  noise  ratio. 


SIGNAL  TO  NOISE  RATIO  (dB) 

Fig.  B5:  Optimal  Eigenvector  Method 


Probability  of  the  peak-level  of  a  single  source.  Parameter  is  input  signal-lo-noise  ratio. 


SIGNAL  TO  NOISE  RATIO  (dB) 

Fig.  B 7:  Schmidt  Eigenvector  Method 


Probability  of  the  peak-level  of  a  single  source.  Parameter  is  input  signal-to-noise  ratio. 
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APPENDIX  C 


Summary  of  the  two-source  simulations 

This  appendix  contains  the  results  of  the  two-source  simulations.  The  approach  that  was 
used  to  plot  the  bearing  estimation  is  illustrated  in  the  following  figure: 


a.  Every  experiment  m  which  the  two  sources  were  resolved  has  been  used  to 
estimate  the  bearings  of  the  sources. 

b.  The  ensemble  of  all  the  successful  experiments  yielded  the  distribution  of 
bearing  estimations. 

c.  The  distributions  have  beev  plotted  cumulatively  so  that  one  can  read  directly 
the  probability  of  the  individual  bearing  estimate  being  closer  to  the  common 
bearing  centre.  Ignoring  the  failed  experiments  results  in  a  plateau  between 
the  two  cumulative  distributions.  The  ordinate  value  at  which  the  plateau  is 
found  is  nothing  more  than  the  probability  that  the  resolution  experiment  will 
fail. 

Concept  of  data  presentation 


Figures  Cl  to  C7  present  the  cumulative  distributions  for  all  seven  beamformers.  The 
probability  axis  (ordinate)  is  again  scaled  such  that  normal  distributions  result  in  a  straight 
line.  The  abscissa  measures  the  separation  in  beam  numbers  (cosine  scale).  Again  1  unit 
corresponds  to  the  reciprocal  aperture  and  the  source  separation  is  given  as  parameter. 
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As  expected,  the  bearing  estimates  are  normally  distributed  only  for  wide  separations. 
As  the  two  sources  close,  the  gaussian  behaviour  of  the  distribution  is  lost,  resulting  in 
long  tails  towards  the  common  centre  bearing,  and  the  two  sources  start  to  interact.  This 
interaction  results  in  resolution  failure  when  the  sources  are  very  close. 

The  non-gaussian  behaviour  at  close  separations  shows  also  that  simple  second  order  statis¬ 
tics  cannot  properly  describe  the  resolution  performance.  However  the  figures  may  be  used 
to  check  the  bias  by  comparing  the  simulated  separation  with  the  distance  of  the  medians; 
the  sensitivity  of  the  bearing  estimation  inay  be  found  by  taking  the  derivative  of  the 
cumulative  distribution;  and  the  resolution  probability  can  be  read  directly. 
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